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Overview 


= UL 4600 standard for AV safety cases 
e Fully autonomous vehicles 
e Issued April 2020 
e How to contribute to the next version 


m Key 4600 ideas: 
e System-level safety case provides direction 
e Vehicle as well as infrastructure and lifecycle processes all matter 
e Safety metrics used for feedback loops 
e Third party component interface protects proprietary info 
e 4600 helps you know that you've done enough work on safety 
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EDGE CASE 
Goal Based Approach © RESEARCH 


= Traditional safety standards are prescriptive 


e “Here is how to do safety” (process, work products) 
— ISO 26262, ISO/PAS 21448, IEC 61508, MIL-STD 882, etc. 


= UL 4600 is goal based 


e “Here is what a safety case should address” 
—- Do NOT prescribe any particular engineering approach 
» Use other safety standards within the safety case context 
e Standard for how to assess a safety case 
—- Minimum coverage requirement (what goes in the safety case?) 
— Properties of a well-formed safety case 
— Objective assessment criteria 
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Example 4600 Clause CB teeseancr 


12.3.1 V&V shall provide acceptable coverage of safety related faults associated with the design phase. 
12.3.1.1 MANDATORY: 

a) Systematic design defects 

b) Design consideration of faults, corruption, data loss, and integrity loss in sensor data 

c) Requirement gaps/omissions and requirement defects 

d) Response to violation of requirement assumptions 

EXAMPLE: Response to exceptional operational environment 

e) Identification and description of the intended ODD 

f) Acceptable mitigation of aspects of the defined fault model for each component and other aspect of the item 
12.3.1.2 REQUIRED: 

a) Maintenance procedure definitions 

NOTE: While maintenance occurs during the lifecycle, the definition of procedures needs to correspond to design 

requirements and assumptions made in design regarding maintenance. 

b) Operational procedure definitions (including startup and shutdown) and operational modes 

c) Faults, corruption, data loss, and integrity loss in data from external sources 

d) Faults and failures associated with exceptional conditions that impair risk reduction functionality 

e) Hardware and software errata and other third-party component design defects 

f) Other faults in safety related functions, component designs, and other designed properties 
12.3.1.3 HIGHLY RECOMMENDED —-N/A 
12.3.1.4 MAGLI SU N/A 
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e EDGE CASE 
Flexible Approaches RESEARCH 
6.4.1 Each identified hazard shall be given a criticality level and assigned an initial risk assuming the 
absence of mitigation. 
6.4.1.1 MANDATORY: 
a) Hazard Log records criticality level and initial risk for each hazard 
6.4.1.2 REQUIRED: 
a) Use of at least one of the following risk evaluation approaches: 
1) Risk table 
2) Risk equation (weighted probability times severity) 
) Fault Tree Analysis (FTA) 
) Event Tree Analysis (ETA) 
) Preliminary Item Safety Assessment(PSSA) 
) Hazard Analysis and Risk Assessment (HARA) 
) Bowtie diagram 
) System-Theoretic Accident Model and Processes (STAMP) 
9) Field engineering feedback 
10) Other relevant risk evaluation approaches 
b) Use of integrity level and related techniques 
EXAMPLES: Integrity level and related techniques from ISO 26262, IEC 61508; development assurance level from DO-178 
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6.4.1.3 HIGHLY RECOMMENDED: 
a) Use of integrity levels defined in an accepted domain-relevant functional safety standard 
NOTE: It might not be practical to use such integrity levels for all aspects of an autonomous systems, but it is highly 
recommended to do so to the extent reasonable. 
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Safety Case 


EDGE CASE 
© RESEARCH 
= Claim — a property of the system 


e “System avoids pedestrians’ CLAIM 


= Argument —- why this is true 





e “Detect & maneuver to avoid” SaaS 
m Evidence — supports argument ; : al 
e Tests, analysis, simulations, ... 
= Sub-claims/arguments address 
complexity 


e “Detects pedestrians’ // evidence 
e “Maneuvers around detected pedestrians’ // evidence 
e “Stops if can't maneuver’ // evidence 
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4600 Safety Case Scope CB eseance 


a Everything needed to independently assess safety 
e Hazards and mitigation approaches 
e Claims traced: arguments to evidence 








=m Scope includes: pee 
Technology: HW/SW, machine learning, als, 

Lifecycle: deployment, operation, incidents, ee ef 2 
Infrastructure: vehicle, roads, data networks, cloud computing, ... 
Road users: pedestrians, light mobility, emergency responders, ... 
Environment: Operational Design Domain (ODD) definition 

.. and more ... 
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Example ODD Prompts (§8.2.2) 


Behavioral rules ligt) 2 Dep 
e EXAMPLES: Traffic laws, vehicle path conflict resolution i pe ie * et 
priority, local customs, justifiable rule breaking for safety ; ; | 
Compliance strategy of traffic rules and regulations y ee f= ot 
e EXAMPLE: Enumeration of applicable traffic regulations and LEE ; | | 
corresponding ego vehicle behavioral constraints https://bit.ly/2IKIZJ9 


Vulnerable populations including number, density, and types 
e EXAMPLES: Pedestrians, motorcycles, bikes, scooters, other vulnerable road users, other road users 


Special road user rules, if applicable 


e EXAMPLES: Bicycles, motorcycles, lane splitting, interacting with construction vehicles, oversize 
vehicles, snowplows, sand/salt trucks, emergency response vehicles, street sweepers, horse-drawn 
vehicles 


Seasonal effects 


e EXAMPLES: Foliage changes (e. g., leaves (dis) appearing), sun angle changes, seasonal behavioral 
patterns (e. g., summer beach traffic), seasonally-linked events (Oktoberfest, regatta crowds, fireworks 


gatherings, air shows) 
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RESEARCH 


| SPI Metrics 
= Safety Performance Indicator (SPI) 
e Like a KPI, but specific to safety 


e Provides metrics on safety case validity 








= SPI measures: 


e Behavior metrics for safety-related behaviors 
-— E.g.: Acceptable violation rate of standoff to pedestrians 
e Assumption validity within safety case 
— E.g.: Tolerates gaps of up to X meters in lane markings 
— E.g.: Correlated camera and lidar false negative rate 
e Any other metrics that validate safety case 
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Feedback Loops 


= Rather than assume perfection... 


.. Manage & improve imperfections 
e Feedback data incorporated in safety case 
e Convert “unknowns” into “knowns’ over time 


= Feedback loops for continuous improvement 
e Implementation faults 
e Design faults 
e Gaps in simulations, analysis tools, ... 
e Gaps in Operational Design Domain 
e Gaps in machine learning training data 
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Elements out of Context (EooC) 


=m Reused or 3'¢ party system “component” 
e Similar in spirit to ISO 26262 SEooC 
e Hardware, software, sensor, map data, ... 
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CLAIM 
m EooC has a safety case fragment [sa 

e Vendor need not expose that safety case _ 

e Instead, provides an interface containing: : 
- Properties & characteristics #9 —~@— ——x____. 
— Assumptions that system must honor -SuARGUMENT?A | o0°°%. [oot inroce 
— Fault model used for assessment {SS (swvancunenr 28 
— 4600 clause coverage (might be partial) £ 


EVIDENCE 2B 
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— Assessment report 





Complementing Other Standards G RESEARCH 


= ISO 26262, MIL-STD 882, etc.: potential starting points 
e Still useful where applicable 


= ISO/PAS 21448 etc. for scenarios re. 
e Design and validation process framework fie a 
e SaFAD and emerging standards 


= 4600 has #DidYouThinkof That? lists 
e Initial safety case coverage 
e Learn from experience: yours; others 
e Objective assessment criteria for safety case 
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° EDGE CASE 
Other Key Points RESEARCH 
= Self-certification is permitted 

e Internal assessor permitted; no external “certificate” requirement 


=m Only necessary technical mitigations required 





e “Does not apply to this system” and “Outside ODD” are OK mihi 
e Can use non-technical mitigations erinpieer eee cecery 
= Underwriters Laboratories is a non-profit SDO eoisiistian oeaamnocnae eon 


e Voting committee (STP) has diverse representation 
e Continuous Maintenance process provides timely updates 

= Does 4600 conflict with ISO 26262 or ISO/PAS 21448? 
e No 

= What if you cant afford to buy a copy? 


e Issued standard is free to browse (“digital view”) on-line in its entirety: 
https://www.shopulstandards.com/ProductDetail.aspx?productid=UL4600 
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Review of Key Ideas 


= System-level safety case provides direction 
e Highlights gaps in evidence and arguments 
= Vehicle, infrastructure, and lifecycle processes all matter 
e If safety case depends upon it, that makes it safety related 
= Metrics combine with feedback loops 
e Operational feedback will be essential for practical safety 
= Third party component interface to protect proprietary info 
e EooC interface permits separate component assessment 
= 4600 helps you know that you've done enough safety work 
e Robust prompts and pitfalls capture best practice/lessons learned 
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Next Steps 


= 4600 provides: 
e Guidance on building safety case 
e Robust minimum criteria | | 
e Emphasis on ability to assess validity scone 


for Safety for the Evaluation of Autonomous Products—it is the first Standard addressing autonomous 
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vehicles and other applications. 


Free Digital View Version (requires registration) 


Standard Technical Panel (STP) Industry Member List 


m= You can get involved! 
e More info on 4600: —— aa 


Cg Witersrorsrente ‘for Rea . 
General Stakeholder 
~/ Ri 


— https://edge-case-research.com/ul4600/ oe. , 
e Teams already working toward adoption a jaa 


Deborah Prince, Underwriters Laboratories 
Research 


discussion about UL 4600: Standard for Safety overview of UL 4600: Standard for Safety for the 


for the Evaluation of Autonomous Products. Evaluation of Autonomous Products. This 


e Participate in the 2020 update cycle ee ee 
— Stakeholders can submit comments (free) 
— Register with: Deborah.Prince@ul.org 


with ISO Standards, and other topics. 
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